Linear Neural Networks Revisited: From PageRank to Family Happiness

The study of Artificial Neural Networks started with the analysis of linear neurons. It was then discovered that networks consisting only of linear neurons cannot describe non-linear phenomena. As a result, most currently used neural networks consist of non-linear neurons. In this paper, we show that in many cases, linear neurons can still be successfully applied. This idea is illustrated by two examples: the PageRank algorithm underlying the successful Google search engine and the analysis of family happiness. 1 Linear Neural Networks: A Brief Reminder Neural networks. A general neural network consists of several neurons exchanging signals. At each moment of time, for each neuron, we need finitely many numerical parameters to describe the current state of this neuron and the signals generated by this neuron. The state of the neuron at the next moment of time and the signals generated by the neuron at the next moment of time are determined by its current state and by the signals obtained from other neurons. Non-linear and linear neural networks. In general, the dependence of the neuron’s next state and/or next signal on its previous state and previous signals is non-linear; see, e.g., [5]. However, original neural networks started with linear neurons, and, as we will argue, there are still cases when linear neurons work well. What we do in this paper. In this paper, we show that there are indeed many useful applications of linear neural networks.